Local dynamics in trained recurrent Neural networks - Supplemental Material
نویسندگان
چکیده
The main text assumes stability of the open loop system, both to guarantee the existence of the xed point x̄, and to justify the application of the Nyquist criterion. Here we argue that this condition holds for networks trainable by Echo State protocol [1] or by FORCE [2]. The terms "echo state property" and "fading memory property" coined in [1] refer to the requirement for globally convergent dynamics of the reservoir network. This property was originally claimed to be a necessary condition for reservoir network training to succeed. However, Sussillo and Abbott [2] showed that intrinsically chaotic networks (g > 1) can be successfully trained as well. In principle, there are two hypothetical mechanisms by which a rank one perturbation wFBw T out, added to a stochastic matrix W with spectral radius greater than unity, might enable the formation of a stable trajectory:
منابع مشابه
Local Dynamics in Trained Recurrent Neural Networks.
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task-related neural dynamics, we study trained recurrent neural networks. We develop a mean field theory for reservoir computing networks trained to have multiple fixed point attractors. Our main result is that the dynamics of the network's output in the vicinity of attractors is gover...
متن کاملIntelligent identification of vehicle’s dynamics based on local model network
This paper proposes an intelligent approach for dynamic identification of the vehicles. The proposed approach is based on the data-driven identification and uses a high-performance local model network (LMN) for estimation of the vehicle’s longitudinal velocity, lateral acceleration and yaw rate. The proposed LMN requires no pre-defined standard vehicle model and uses measurement data to identif...
متن کاملMulti-Step-Ahead Prediction of Stock Price Using a New Architecture of Neural Networks
Modelling and forecasting Stock market is a challenging task for economists and engineers since it has a dynamic structure and nonlinear characteristic. This nonlinearity affects the efficiency of the price characteristics. Using an Artificial Neural Network (ANN) is a proper way to model this nonlinearity and it has been used successfully in one-step-ahead and multi-step-ahead prediction of di...
متن کاملIdentifying Fixed Points in Recurrent Neural Networks using Directional Fibers: Supplemental Material on Theoretical Results and Practical Aspects of Numerical Traversal
Fixed points of recurrent neural networks can represent many things, including stored memories, solutions to optimization problems, and waypoints along non-fixed attractors. As such, they are relevant to a number of neurocomputational phenomena, ranging from low-level motor control and tool use to high-level problem solving and decision making. Therefore, global solution of the fixed point equa...
متن کاملAnalysis and Visualization of the Dynamics of Recurrent Neural Networks for Symbolic Sequences Processing
Recurrent neural networks unlike feed-forward networks are able to process inputs with time context. The key role in this process is played by the dynamics of the network, which transforms input data to the recurrent layer states. Several authors have described and analyzed dynamics of small sized recurrent neural networks with two or three hidden units. In our work we introduce techniques that...
متن کامل